A presentation for Research Data at the Edge, Day One of Duke Research Computing Symposium

Outline

• API
• JSON
• R / RStudio

Why?

• The Web has lots of stuff
  • frontier beyond curated datasets
  • stuff is wrapped in HTML
  • The HTML is transported over HTTP but composed for h2m consumption
  • Intellectual Property Rights bear serious consideration

API

Application Program Interface

• Built for machine-to-machine interactions
• Instructions for programs

Client / Server

• Make [R] interface with the web, same as h2m but now m2m

Human Simulation

• Human version
  • Person uses Web Client
  • Person enters a URL
    
  • client & server negotiate
    
  • Information is sent back in wrapped HTML
  • Web Browser parses the HTML

m2m – development

dramatization: confused about the protocol

JSON

• Javascript Object Notation is a language-independent data format
• Currently the most common data data format for asynchronous client/server communication format
• Consists of key-value pairs

# from https://en.wikipedia.org/wiki/JSON
{
  "firstName": "John",
  "lastName": "Smith",
  "isAlive": true,
  "age": 25,
  "address": {
    "streetAddress": "21 2nd Street",
    "city": "New York",
    "state": "NY",
    "postalCode": "10021-3100"
  },
  "phoneNumbers": [
    {
      "type": "home",
      "number": "212 555-1234"
    },
    {
      "type": "office",
      "number": "646 555-4567"
    },
    {
      "type": "mobile",
      "number": "123 456-7890"
    }
  ],
  "children": [],
  "spouse": null
}

Example

OMDB api

like http://imdb.com/

R Packages – Related

People who use JSONlite also use…

• httr – calls JSONlite in service to major goal of managing HTTP
• rvest – used for html parsing